The invention relates generally to digital image processing and, more particularly, to the modification of digital images.
Recent advances in automation utilize artificial intelligence to identify objects and to provide information relating to the position, shape, orientation, and other characteristics of the object.
The "eye" of the artificial intelligence system is a video camera and processor that generates a digital image comprising a series of frames. Each frame is comprised of pixels which are the smallest area unit of the image. The location of a pixel in the frame is indicated by its x,y coordinates. The y coordinate identifies the line of the frame and the x coordinate identifies the position in the line of each pixel.
Each pixel in the frame is assigned a digital value by the image processing system. These digital values are utilized in subsequent processing.
Once the "eye" of the system has formed an image, the "brain" of the system must interpret the image. One method of interpreting the image is to store in memory some selected characteristics of a reference object. The video system then analyzes the video image to determine the selected characteristics of objects in the image. These determined characteristics of the objects and the image may then be compared to the stored characteristics of the reference object.
For example, if the selected characteristic were the projection of the reference object on the x axis of the frame, then a reference value of this projection is stored in memory. The determined value of the projection of an object in an image is supplied to a processor. The processor then compares the projection of the referenced object and the imaged object and decides whether the objects are identical.
Unfortunately, most image sensors in cameras do not produce perfect images. The imperfections can arise from many sources. Dirt, highlights, and distortions in the air or lens optics may distort the image before it ever arrives at the sensor. The sensor itself will add further distortions, since the individual pixels of the area array may have slightly different response characteristics.
A common type of distortion is uneven response of different parts of the sensor. For example, the images produced might always seem brighter towards the center of the image. To correct the image, a separate frame buffer that holds a custom gain and offset for each pixel of the sensor may be used. As each pixel is brought in, it is "boosted" by multiplying the pixel value by a value stored in the correction frame buffer, and "stretched" by subtracting from the pixel valve another value stored in the correction frame buffer.
If the image signal is very low, one may have a low signal to noise ratio. To correct this, one must boost the signal. One method is to average together the images of several frames. This is accomplished by adding successive frame data together. As many frames as desired can be summed together, with two through eight frames being the normal range. An even more complex averaging allows one to separately weigh the images as they are summed together. This is accomplished by multiplying the pixel in the given frame by a constant value before it is summed with the pixels in the other frames.
When matching an image, portions of the input image are compared with portions of the reference image in storage. For example, the system will compare all rectangular corners in the reference sample with those in the input image. For real time throughputs, one may not have time to look everywhere in fine detail for the rectangles. To facilitate matching, it usually is necessary to first average down the image to half size (or less) to reduce the search space. After the most likely candidates have been found, they can then be re-examined at higher resolution. For a half-size image reduction the output frame is formed by adding together four neighboring pixels. That is, a pixel is added to the pixel one line above it on the screen, and the pixels directly adjacent these two pixels are added to the sum.
Existing ALUs perform simple functions such as addition, subtraction, boolean operations, and generation of ones or zeros. When processing images as described above, it is sometimes necessary to determine the maximum of two variables, the absolute value of a difference of two values, and to perform such functions as clipping addition (x+y=x+y or 65535, whichever is smaller) zero-clipping subtraction (x-y=x-y or zero, whichever is greater) and other complex functions which simple ALUs are incapable of performing.